# Implementation Plan
This document outlines a phased approach to developing the **cli_engineer** utility. Each phase includes a checklist of tasks to ensure all components are implemented, tested, and integrated for non-interactive, automated execution. Developers can mark tasks as complete to track progress.
**Note**: The initial implementation has established a basic agentic loop structure. This plan has been updated to reflect the current state and guide the remaining development.
---
## Current Status (Last Updated: 2025-05-31)
### Completed
- **Foundation Phase**: Basic CLI structure, Rust 2024 edition, core modules
- **Configuration System**: TOML-based config with serde, default values, CLI arg merging
- **Event Bus**: Component communication, metrics tracking, event emission
- **AI Provider Framework**: OpenAI provider implementation, provider trait system
- **Enhanced UI Foundation**: Added dependencies (indicatif, colored, crossterm)
- **Crate Publication Setup**: All metadata configured for crates.io
### In Progress
- **Enhanced Terminal UI**: Created ui_enhanced module with colors and progress bars
- **Artifact Management**: Need to implement artifact creation and tracking
- **Context Management**: Need to implement context compression and caching
### To Do
- Real implementation of planner, executor, reviewer modules
- Anthropic provider completion
- MCP (Model Context Protocol) integration
- GitHub integration
- Parallel task execution
- Semantic linting
- Visual analysis integration
- Testing suite expansion
---
## Phase 1: Foundation Setup
Establish the core infrastructure, including CLI, configuration, and event bus.
- [x] Initialize the Rust project with `cargo new cli_engineer`.
- [x] Add dependencies (`clap`, `tokio`, `anyhow`, `async-trait`, `log`, `simplelog`).
- [x] Implement CLI structure using `clap`.
- [x] Develop TOML-based configuration management.
- [x] Implement the event bus using Rust channels or `bus` crate.
- [x] Write basic CLI command parsing.
- [x] Add dependency for `serde`, `serde_toml` for configuration.
- [x] Add dependency for `bus` or implement custom event system.
- [x] Write unit tests for configuration loading.
- [x] Write unit tests for event bus functionality.
**Current State**: Basic CLI with headless/verbose flags and command input is implemented. Need to add configuration file support and event bus.
---
## Phase 2: AI Integration
Integrate AI providers with prompt optimization and model switching.
- [x] Define an abstract trait for AI providers (`LLMProvider`).
- [x] Implement a dummy provider (`LocalProvider`) for testing.
- [x] Implement real AI providers:
- [x] OpenAI (`reqwest`, `serde_json` dependencies needed)
- [ ] Anthropic
- [ ] Gemini
- [ ] xAI
- [ ] OpenRouter
- [ ] Ollama
- [x] Add configuration for selecting providers and models.
- [ ] Implement model switching logic based on task requirements.
- [ ] Implement prompt optimization for precise code queries.
- [ ] Track and emit API cost metrics via event bus.
- [ ] Add retry logic and error handling for API calls.
- [ ] Write unit tests for each provider.
- [ ] Write tests for model switching.
- [ ] Write tests for prompt optimization accuracy.
**Current State**: LLMProvider trait and LocalProvider stub exist. LLMManager manages providers but lacks real implementations.
---
## Phase 3: Task Orchestration Enhancement
Enhance the current agentic loop with proper task orchestration.
- [x] Basic agentic loop structure exists.
- [ ] Refactor to match architecture's Task Orchestrator design:
- [ ] Move from simple loop to event-driven orchestration
- [ ] Add task prioritization
- [ ] Support automated workflow based on configurations
- [ ] Implement proper task types beyond simple text descriptions.
- [ ] Add support for task metadata and configuration.
- [ ] Integrate with event bus for progress tracking.
- [ ] Add task persistence for resumable workflows.
**Current State**: Basic interpreter → planner → executor → reviewer loop exists but needs enhancement.
---
## Phase 4: Codebase Interaction
Enable automated agentic search, file operations, and shell command execution.
- [ ] Implement semantic code search with fuzzy matching and cross-file analysis.
- [ ] Add dependencies: `walkdir`, `regex`, `similar` for code search.
- [ ] Implement automated file operations (create, read, update, delete) with AI validation.
- [ ] Implement safe shell command execution using `std::process::Command`.
- [ ] Add command sandboxing for security.
- [ ] Capture and parse command outputs.
- [ ] Integrate with the event bus for command output events.
- [ ] Write unit tests for semantic search.
- [ ] Write unit tests for file operations.
- [ ] Write unit tests for shell command execution.
- [ ] Write security tests for command injection prevention.
---
## Phase 5: Artifact and Execution Management
Build artifact creation, execution environments, and validation.
- [ ] Design artifact storage system (file system-based).
- [ ] Add dependency for `tempfile` for isolated environments.
- [ ] Implement artifact creation and storage logic.
- [ ] Set up isolated execution environments:
- [ ] Python virtualenv support
- [ ] Container support (optional)
- [ ] Develop dependency installation logic (AI-driven or explicit).
- [ ] Implement code execution and output capture.
- [ ] Implement execution planning for complex tasks.
- [ ] Implement artifact validation (e.g., syntax, linting).
- [ ] Create JSON artifact manifest with metadata.
- [ ] Add dependency for `syntect` or similar for syntax validation.
- [ ] Integrate with the event bus for progress/completion events.
- [ ] Write comprehensive tests for all components.
---
## Phase 6: Context Management
Implement context tracking, compression, and caching.
- [ ] Implement context tracking for each AI model.
- [ ] Add token counting logic (provider-specific).
- [ ] Develop summarization logic for context compression.
- [ ] Add logic to trigger compression at 50% usage.
- [ ] Implement context caching for frequent snippets.
- [ ] Add dependency for `tiktoken-rs` or similar for token counting.
- [ ] Integrate with the event bus for context usage metrics.
- [ ] Write comprehensive tests.
---
## Phase 7: Terminal UI Enhancement
Upgrade from basic spinner to full terminal UI.
- [x] Basic spinner UI implemented.
- [ ] Add dependency for `indicatif` for progress bars.
- [ ] Add dependency for `colored` or `ansi_term` for colorful output.
- [ ] Implement ANSI escape code rendering for sections.
- [ ] Design layout for artifacts, execution, and metrics display.
- [ ] Create real-time metric displays:
- [ ] Context usage gauge
- [ ] API cost tracker
- [ ] Execution timer
- [ ] Implement artifact manifest table view.
- [ ] Subscribe to event bus for UI updates.
- [ ] Add support for different output formats (JSON, plain text).
**Current State**: Simple spinner implemented. Needs full UI upgrade per requirements.
---
## Phase 8: MCP and Visual Analysis
Integrate MCP servers and visual analysis with web content retrieval.
- [ ] Research and understand MCP protocol.
- [ ] Implement MCP client protocol.
- [ ] Add dependency for WebSocket support if needed.
- [ ] Integrate with Playwright MCP server for UI screenshots.
- [ ] Set up communication with visual analysis models.
- [ ] Implement web content retrieval.
- [ ] Add dependency for `scraper` for web parsing.
- [ ] Process analysis results and incorporate into workflows.
- [ ] Write comprehensive tests.
---
## Phase 9: Quality and Collaboration
Add semantic linting, refactoring, and GitHub integration.
- [ ] Develop AI-based semantic linting logic.
- [ ] Implement customizable linting rules.
- [ ] Add rule configuration system.
- [ ] Implement code refactoring suggestions.
- [ ] Integrate with GitHub API:
- [ ] Add dependency for `octocrab` or similar.
- [ ] Issue management
- [ ] PR creation and updates
- [ ] Code review automation
- [ ] Write comprehensive tests.
---
## Phase 10: Parallel Task Handling
Enable concurrent tasks with prioritization.
- [x] Basic concurrency module exists.
- [ ] Enhance concurrent execution with proper task management.
- [ ] Develop logic for task prioritization.
- [ ] Implement result synthesis from parallel tasks.
- [ ] Add proper error handling for parallel failures.
- [ ] Integrate with event bus for task progress/results.
- [ ] Write comprehensive tests.
**Current State**: `concurrency.rs` exists with `run_parallel` function but is unused.
---
## Phase 11: Testing and Documentation
Comprehensive testing and documentation.
- [ ] Set up integration test framework.
- [ ] Add code coverage tracking.
- [ ] Performance profiling and optimization.
- [ ] Write user documentation.
- [ ] Write developer/API documentation.
- [ ] Create example configurations and use cases.
- [x] Basic README exists.
- [ ] Enhance README with detailed usage examples.
---
## Phase 12: Packaging and Distribution
Finalize for distribution.
- [x] Basic Cargo.toml configured for crates.io.
- [x] MIT License included.
- [ ] Set up GitHub Actions for CI/CD.
- [ ] Add pre-commit hooks for code quality.
- [ ] Create binary releases for major platforms.
- [ ] Write installation guides.
---
## Implementation Notes
### Key Architecture Differences from Current Implementation
1. **Event-Driven vs. Direct Calls**: The architecture specifies an event bus for component communication, but the current implementation uses direct method calls.
2. **Task Orchestrator**: Currently implemented as `AgenticLoop`, but needs to evolve into a proper orchestrator with configuration-driven automation.
3. **Missing Components**:
- Configuration management
- Event bus
- Real AI providers
- Artifact management
- Codebase interaction
- Enhanced terminal UI
- MCP integration
- GitHub integration
### Recommended Next Steps
1. **Add Configuration System**: Implement TOML-based configuration to drive automated workflows.
2. **Implement Event Bus**: Create the event-driven architecture specified in the design.
3. **Add Real AI Provider**: Start with OpenAI as the first real provider implementation.
4. **Enhance Terminal UI**: Upgrade from spinner to full colorful display with metrics.
### Dependencies to Add
```toml
[dependencies]
# Already included
tokio = { version = "1.36", features = ["full"] }
thiserror = "1.0"
log = "0.4"
simplelog = "0.12"
anyhow = "1.0"
async-trait = "0.1"
clap = { version = "4.5", features = ["derive"] }
# To be added
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
toml = "0.8"
reqwest = { version = "0.11", features = ["json"] }
indicatif = "0.17"
colored = "2.0"
bus = "2.4"
walkdir = "2.4"
regex = "1.10"
similar = "2.4"
tempfile = "3.8"
syntect = "5.1"
tiktoken-rs = "0.5"
scraper = "0.18"
octocrab = "0.32"
```
---
**Usage Note**: This plan reflects the current implementation state and provides a roadmap for completing the full vision outlined in the PRD and Architecture documents.